Model-based testing of a highly programmable system

The paradigm of model-based testing shifts the focus of testing from writing individual test cases to developing a model from which a test suite can be generated automatically. We report on our experience with model-based testing of a highly programmable system that implements intelligent telephony services in the U.S. telephone network. Our approach used automatic test-case generation technology to develop sets of self-checking test cases based on a machine-readable specification of the messages in the protocol under test. The AETGTM software system selected a minimal number of test-data tuples that covered pairwise combinationsof tuple elements. We found the combinatorial approach of covering pairwise interactions between input fields to be highly effective. Our tests revealed failures that would have been difficult to detect using traditional test designs. However, transferring this technology to the testing organization was difficult. Automatic generation of cases represents a significant departure from conventional testing practice due to the large number of tests and the amount of software development involved.